Information processing apparatus, method and computer-readable medium

ABSTRACT

In one example embodiment, an information processing apparatus causes a display device to display a first image from images associated with an observation target object. The images include the first image and a second image which corresponds to an annotation mark. In this embodiment, the information processing apparatus also causes the display device to display the annotation mark corresponding to the second image. In this embodiment, the displayed annotation mark overlaps the first image.

CROSS REFERENCES TO RELATED APPLICATIONS

The present application is a continuation of U.S. application Ser. No.12/899,726, filed on Oct. 7, 2010, which claims priority to JapanesePatent Application No. JP 2009-272907, filed in the Japanese PatentOffice on Nov. 30, 2009, and Japanese Patent Application No. JP2010-189614, filed in the Japanese Patent Office on Aug. 26, 2010, theentire content of each of which is hereby incorporated by referenceherein.

BACKGROUND

In the field of medicine, pathology, or the like, there has beenproposed a system that digitizes an image of a cell, a tissue, an organ,or the like of a living body, that is obtained by an optical microscope,to examine the tissue or the like by a doctor or a pathologist or todiagnose a patient based on the digitized image.

For example, Japanese Patent Application Laid-open No. 2009-37250(hereinafter, referred to as Patent Document 1) discloses a method inwhich an image optically obtained by a microscope is digitized by avideo camera with a CCD (charge coupled device), a digital signal isinput to a control computer system, and the image is visualized on amonitor. A pathologist performs examination while watching the imagedisplayed on the monitor (see, for example, paragraphs [0027] and [0028]and FIG. 5 of Patent Document 1).

Further, Japanese Patent Application Laid-open No. 2000-148895(hereinafter, referred to as Patent Document 2) discloses a system inwhich position information is set on an arbitrary portion in such animage, for example, on a portion which may have a disorder, and contentinformation is set in relation with the position information. In thissystem, a mark as the position information is displayed on the image, aselecting operation with respect to the displayed mark by using an inputapparatus such as a mouse is accepted, and a comment as the contentinformation registered in relation with the position information isdisplayed (see, for example, paragraph [0027] and FIG. 5 of PatentDocument 2). The function in which position information or contentinformation is set in relation with an image, and an image visuallycompounded with the position information or the content information isdisplayed is referred to as an “annotation function.”

Meanwhile, in recent years, there is proposed an apparatus capable ofdisplaying image data of one observation target object, the image databeing of a plurality of positions different in an optical axis directionof a microscope, such that the image data are continuously displayed inthe order of the optical axis direction (see, for example, paragraph[0023] and FIG. 3 of Japanese Patent Application Laid-open No.2008-500643) (hereinafter, referred to as Patent Document 3). Herein, aset of image data of one observation target object, the image data beingof a plurality of positions different in an optical axis direction of amicroscope, is referred to as “Z-stack”. According to such an apparatus,by continuously displaying, in an optical axis direction, image data ofa plurality of positions different in the optical axis direction, a usercan get a feeling of observing images while changing focuses of amicroscope.

In a case where an image on which position information with regard to anannotation is set and an image on which position information is not setare mixed in a Z-stack, an image including a mark, which is obtained byvisually compounding the position information of the annotation, and animage including no mark are continuously displayed in a mixed manner ona display screen. As a result, there is a fear that an observer may missthe mark and therefore may miss a disorder. For example, in thecontinuing display, owing to speed of changing images by an observer,the observer may miss a mark included in an image locating at the middleof the images.

In view of the above-mentioned circumstances, it is desirable to providean information processing apparatus, a method and computer-readablemedium for avoiding a risk of missing an annotation by an observer.

SUMMARY

The present disclosure relates to an information processing apparatus, amethod, and a computer-readable medium for controlling display of imagedata obtained by a microscope in a field of medicine, pathology,biology, materials science, or the like.

In one example embodiment, an information processing apparatus includesa processor and a memory device operatively coupled to the processor,the memory device storing instructions that cause the processor, incooperation with the memory device, to cause a display device todisplay: (a) a first image from images associated with an observationtarget object (e.g., a section of biological tissue), the imagesincluding the first image and a second image; and (b) a first annotationmark corresponding to the second image, the first annotation markoverlapping the first image.

In an example embodiment, the images are obtained by a microscope. In anexample embodiment, the first image and the second image have a firstresolution. In an example embodiment, the instructions cause theprocessor to enable a user to change the first resolution of the firstimage and the second image to a second, different resolution.

In an example embodiment, the images correspond to a plurality ofdifferent positions in an optical axis direction. In an exampleembodiment, the plurality of different positions in the optical axisdirection are determined based on focusing positions.

In an example embodiment, the first annotation mark includes apredetermined shaped frame.

In an example embodiment, the instructions cause the processor tooperate with the display device to display an annotation commentoverlaying the displayed first image, the annotation comment beingdifferent from the displayed first annotation mark.

In an example embodiment, an input device is operatively coupled to theprocessor, and the instructions cause the processor to operate with theinput device to enable a user to input annotation information.

In an example embodiment, the first annotation mark corresponds to aposition on the second image, and the instructions cause the processorto cause a display device to display the first annotation mark at aposition on the first image which corresponds to the position of thesecond image.

In an example embodiment, the instructions cause the processor todisplay a second annotation mark corresponding to the first image.

In an example embodiment, the first image corresponds to a first layerwhich corresponds to a first position in an optical axis direction, andthe second image corresponds to a second layer which corresponds to asecond position in the optical axis direction.

In an example embodiment, the first annotation mark and the secondannotation mark have at least one different color, brightness, chroma,alpha blend values, widths of lines, and styles of lines, and theinstructions cause the processor to change at least one of the color,the chroma, the alpha blend values, the widths of lines, and the stylesof lines based on said layers which correspond to said positions in theoptical axis direction.

In an example embodiment, the instructions cause the processor to enablea user to select the first annotation mark. In response to the firstannotation mark being selected, the instructions cause the processor tocause a display device to display the second image, wherein the firstannotation mark overlaps the displayed second image.

In an example embodiment, the instructions cause the processor to enablea user to select one of the first annotation mark and the secondannotation mark. In response to the first annotation mark beingselected, the instructions cause the processor to cancel the display ofthe second annotation mark. In response to the second annotation markbeing selected, the instructions cause the processor to cancel thedisplay of the first annotation mark.

In an example embodiment, the instructions cause the processor to, inresponse to the first annotation mark being selected, arrange anddisplay the first annotation mark on a center portion of the displayedsecond image.

In an example embodiment, the instructions cause the processor todetermine whether a user has requested a change from displaying thefirst image to displaying the second image. In response to the changebeing requested, the instructions cause the processor to determinewhether a mode is set. In response to the mode being set, theinstructions cause the processor to cause a display device to displaythe second image, and the first annotation mark overlapping the secondimage.

In an example embodiment, the instructions cause the processor to causea display device to, in response to the mode being set, display a thirdannotation mark overlapping the second image, the third annotation markcorresponding to a third image.

In an example embodiment, the instructions cause the processor to causethe display device to, in response to the mode not being set, displaythe second image, and the first annotation mark overlapping the secondimage.

In an example embodiment, the instructions cause the processor to, foreach displayed image associated with the observation target object,enable a user to register annotation information in a control table.

In an example embodiment, a method includes of operating an informationprocessing apparatus including instructions includes: (a) causing aprocessor to execute the instructions to cause a display device todisplay a first image from images associated with an observation targetobject, the images including the first image and a second image; and (b)causing the processor to execute the instructions to cause the displaydevice to display an annotation mark corresponding to the second image,the annotation mark overlapping the first image.

In an example embodiment, a computer-readable medium stores instructionsstructured to cause an information processing apparatus to: (a) cause adisplay device to display a first image from a plurality of differentimages associated with an observation target object, the plurality ofimages including the first image and a second image; and (b) cause thedisplay device to display an annotation mark corresponding to the secondimage, the annotation mark overlapping the first image.

In an example embodiment, an information processing apparatus includes aprocessor, and a memory device operatively coupled to the processor, thememory device storing instructions that cause the processor, incooperation with the memory device, to cause a display device todisplay: (a) a first image from a plurality of different imagesassociated with an observation target object, the plurality of differentimages being obtained at a plurality of different focuses, the pluralityof different focuses including a first focus and a second focus, thefirst image being obtained at the first focus, the second focus beingdifferent from the first focus; and (b) an annotation mark correspondingto the second focus, the annotation mark overlapping the first image.

In an example embodiment, a method of operating an informationprocessing apparatus including instructions includes: (a) causing aprocessor to execute the instructions to cause a display device todisplay a first image from a plurality of different images associatedwith an observation target object, the plurality of different imagesbeing obtained at a plurality of different focuses, the plurality ofdifferent focuses including a first focus and a second focus, the firstimage being obtained at the first focus, the second focus beingdifferent from the first focus; and (b) causing the processor to executethe instructions to cause the display device to display an annotationmark corresponding to the second focus, the annotation mark overlappingthe first image.

In an example embodiment, a computer-readable medium stores instructionsstructured to cause an information processing apparatus to: (a) cause adisplay device to display a first image from a plurality of differentimages associated with an observation target object, the plurality ofdifferent images being obtained at a plurality of different focuses, theplurality of different focuses including a first focus and a secondfocus, the first image being obtained at the first focus, the secondfocus being different from the first focus; and (b) cause the displaydevice to display an annotation mark corresponding to the second focus,the annotation mark overlapping the first image.

These and other objects, features and advantages of the presentdisclosure will become more apparent in light of the following detaileddescription of best mode embodiments thereof, as illustrated in theaccompanying drawings.

Additional features and advantages are described herein, and will beapparent from the following Detailed Description and the figures.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram showing an example structure of an informationprocessing system including at least an information processing apparatusaccording to an embodiment of the present disclosure.

FIG. 2 is a diagram showing an example image pyramid structure forexplaining a principle of displaying an image.

FIG. 3 is a diagram for explaining an example procedure at a time whenan image group of the image pyramid structure of FIG. 2 is generated.

FIG. 4 is a functional block diagram showing an example informationprocessing apparatus.

FIG. 5 is a diagram showing an example control table.

FIG. 6 is a flowchart showing an example operation of annotation markdisplay.

FIGS. 7A to 7D are diagrams showing display examples of annotation markdisplay.

FIG. 8 is a diagram showing a display example of annotation commentdisplay.

FIG. 9 is a flowchart showing an example operation of jump display.

FIG. 10 is a diagram showing an display example of jump display.

FIG. 11 is a flowchart showing a modified example of the operation ofjump display.

FIG. 12 is a flowchart showing an example operation of annotationcomment display.

DETAILED DESCRIPTION

Hereinafter, example embodiments will be described with reference to thedrawings.

[Structure of an Example Information Processing Apparatus]

FIG. 1 is a block diagram showing an example structure of an informationprocessing system including at least an information processing apparatusaccording to an embodiment of the present disclosure. As the informationprocessing apparatus, a PC (personal computer) 100 is used, for example.

The PC 100 includes a CPU (central processing unit) 101, a ROM (readonly memory) 102, a RAM (random access memory) 103, an input and outputinterface (hereinafter, abbreviated as I/O interface) 105, and a bus 104that connects those components with one another.

To the I/O interface 105, a display unit 106, an input unit 107, astorage unit 108, a communication unit 109, a drive unit 110, and thelike are connected.

The display unit 106 is a display device that uses liquid crystal, EL(electro-luminescence), a CRT (cathode ray tube), or the like.

The input unit 107 is, for example, a pointing device, a keyboard, atouch panel, or another operation apparatus. In the case where the inputunit 107 includes a touch panel, the touch panel may be integrated withthe display unit 106. Alternatively, the input unit 107 is an inputapparatus connected to the PC and including a dial-type knob.

The storage unit 108 is a non-volatile memory such as an HDD (hard diskdrive), a flash memory, and another solid-state memory.

The drive unit 110 is a device capable of driving a removable recordingmedium 111 such as an optical recording medium, a floppy (registeredtrademark) disk, a magnetic recording tape, and a flash memory. Incontrast, the storage unit 108 is often used as a device that ispreviously included in the PC 100 and mainly drives a recording mediumthat is not removable.

The communication unit 109 is a modem, a router, or anothercommunication apparatus that is connectable to a LAN (local areanetwork), a WAN (wide area network), or the like and is used forcommunicating with another device. The communication unit 109 mayperform either one of a wired communication or a wireless communication.The communication unit 109 is used separately from the PC 100 in manycases.

Next, a description will be given on an image that is obtained by anoptical microscope (not shown) and is mainly stored in the storage unit108 of the PC 100 and on a principle of displaying the image. FIG. 2 isa diagram for explaining image data obtained by an optical microscope(not shown) stored mainly in the storage unit 108 of the PC 100.

An image pyramid structure 50 in this embodiment is an image groupgenerated at a plurality of different resolutions with respect to oneimage obtained from one observation target object 15 (see, FIG. 3) bythe optical microscope. The images are a plurality of image data of oneobservation target object obtained at a plurality of different focusesby the optical microscope. This is called “Z-stack”, which is a functionto deal with the case where tissues or cells may have different shapesalso in the thickness direction of the observation target object 15. Ascanner apparatus has the Z-stack function in many cases, and generatesimages of 5 to 10 layers or 10 to 30 layers. Hereinafter, a descriptionwill be given on the image pyramid structure 50 of one image of aplurality of images.

As shown in FIG. 2, image data obtained from one observation targetobject 15 (see, FIG. 3) by the optical microscope are stored andcontrolled as the image pyramid structure. The image pyramid structureincludes a plurality of image data whose resolutions are reducedstepwise from an original image data, set in a tiered manner. The imagedata of each tier is blocked into areas called “tiles” of apredetermined resolution. In the operations of changing a display rangeof a displayed image or enlarging/reducing the size of an image, byloading required tiles from where they are stored on a display buffer,the operation can be made at high speed. Note that a resolution of onetile is, for example, 256×256 (pixel) or 256×512 (pixel).

Herein, the image pyramid structure of FIG. 2 includes N tiers. The Ntiers are referred to as Nth tier, N-1th tier, N-2th tier, . . . , 0thtier, respectively. The number of tiles in those tiers are 2^(N)×2^(N),2^(N−1)×2^(N−1), 2^(N−2)×2^(N−2), . . . , 1, respectively.

That is, when the display unit 106 displays those images at the samemagnification, for example, 100%, (displays each image by the number ofdots that is physically the same as the number of pixels of the images),the largest image is displayed in the largest size, and the smallestimage is displayed in the smallest size. Here, a display range of thedisplay unit 106 is represented by D in FIG. 2.

FIG. 3 is a diagram for explaining a procedure at a time when the imagegroup of the image pyramid structure 50 is generated.

First, a digital image being an original image is obtained at apredetermined observation magnification by an optical microscope (notshown). The original image corresponds to the largest image which is thelowermost image of the image pyramid structure 50 of FIG. 2. That is,the original image is the image of the highest resolution. Accordingly,as the lowermost image of the image pyramid structure 50, an imageobserved at a relatively high magnification and obtained by an opticalmicroscope is used.

Note that, in the field of pathology, generally, a matter obtained byslicing an organ, a tissue, or a cell of a living body, or a partthereof is an observation target object 15. Then, a scanner apparatus(not shown) having a function of the optical microscope reads theobservation target object 15 stored in a glass slide, to obtain adigital image and store the obtained digital image into the scannerapparatus or another storage apparatus.

The scanner apparatus or a general-purpose computer (not shown)generates, as shown in FIG. 3, from the largest image obtained asdescribed above, a plurality of images whose resolutions are reducedstepwise, and stores those images in unit of “tile” that is a unit of apredetermined size, for example. The size of one tile is 256×256(pixel), for example. The image group generated as described above formsthe image pyramid structure 50, and the storage unit 108 of the PC 100stores the image pyramid structure 50. Actually, the PC 100 only has tostore the images whose resolutions are different with the images beingassociated with resolution information items, respectively. It should benoted that the generating and storing the image pyramid structure 50 maybe performed by the PC 100 shown in FIG. 1.

The PC 100 uses software that employs the system of the image pyramidstructure 50, to extract a desired image from the image pyramidstructure 50 and output the desired image to the display unit 106 inaccordance with an input operation through the input unit 107 by theuser. Specifically, the PC 100 displays an image of an arbitrary partselected by the user, out of the images at an arbitrary resolutionselected by the user. With this operation, the user can get a feeling ofobserving the observation target object 15 while changing theobservation magnification. That is, the PC 100 functions as a virtualmicroscope. A virtual observation magnification in this case correspondsto a resolution in reality.

[Functional Structure of Information Processing Apparatus]

FIG. 4 is a functional block diagram showing the PC 100.

The PC 100 is capable of realizing an annotation function. Herein, theannotation function is a function in which position information orcontent information is set in relation with an arbitrary portion in animage, for example, a portion which may have a disorder, and an imagevisually compounded with the position information or the contentinformation is displayed. Further, a displayed mark obtained by visuallycompounding the position information with the image is referred to as“annotation mark”. A displayed comment obtained by readably compoundingthe content information with the image is referred to as “annotationcomment”.

The PC 100 includes an image obtaining unit 501, an image storing unit502, an annotation setting unit 503 (setting unit), a table storing unit504, an image processing unit 505 (processing unit), a selectedannotation determining unit 506 (selecting unit), a display unit 106,and an input unit 107. These functional units are achieved in a computerresource based on a program loaded on the RAM 103. The image obtainingunit 501 (Obtaining unit) obtains an image via a network or an imagestored in the removable recording medium 111, and stores the obtainedimage in the image storing unit 502. The image storing unit 502 is setin the storage unit 108, the removable recording medium 111, or thelike.

The annotation setting unit 503 (setting unit) causes a user to select adisplay range (for example, one predetermined tile) of an imagedisplayed on the display screen and to select the layer number of theZ-stack, that is, a position and the like in the optical axis direction,and obtains the selecting result. Obtaining the selecting result, theannotation setting unit 503 generates a control table (see, FIG. 5) inassociation with the display range of the image in the table storingunit 504. The table storing unit 504 is set in the RAM 103. Next, theannotation setting unit 503 registers, as the value of the Z number, thevalue of 1 to N each representing the position of the layer of the imageincluding the display range in the control table generated in the tablestoring unit 504. The annotation setting unit 503 registers annotationposition information and annotation content information set through theinput unit 107 by a user in relation with the value of the Z number inthe control table. Operations for setting an annotation on an image of apredetermined layer as described above will be explained later as“operation of annotation setting”.

A user inputs various orders and data through the input unit 107. Forexample, through rotation of the dial-type knob of the input unit 107 bya user, the layer of a displayed image can be changed. For example,through rotation of the dial-type knob of the input unit 107 by a user,images of different layers can be displayed. Accordingly, a user canobserve images while changing the layers of the displayed images as ifthe user observes an observation target object by changing focusesthrough rotation of a focus dial of an actual microscope. Further, byselecting an annotation mark displayed on the display screen through theinput unit 107 by a user, an image of a layer on which the annotation isset can be displayed on the display screen, or a comment representingcontent information of the annotation can be displayed on the displayscreen.

The image processing unit 505 (processing unit) causes a user to selecton/off settings of a Z-through display mode, and obtains the selectingresult. The “Z-through display mode” is a mode in which all theannotation marks set on images of a plurality of layers different in theoptical axis are displayed. The image processing unit 505 obtains anorder to change a layer of a displayed image input through the inputunit 107 by a user, and in accordance with the order, obtains image datastored in the image storing unit 502 or annotation position informationstored in the table storing unit 504. The image processing unit 505compounds the image data obtained from the image storing unit 502 withan annotation mark, which is obtained by visually reflecting theannotation position information, and expands the compound displayinformation in a buffer. Accordingly, an image including an annotationmark is displayed on the display screen. Operations for displaying animage including an annotation mark as described above will be explainedlater as “operation of annotation mark display”.

The selected annotation determining unit 506 (selecting unit) obtains anorder for jump display and an order for displaying an annotation commentinput through the input unit 107 by a user. Herein, each order may beinput with an operation such as a few clicks of the mouse of the inputunit 107 by a user in a state where a cursor is displayed in an areasurrounded by an annotation mark included in an image displayed on thedisplay screen, for example. Obtaining the order for jump display andthe order for displaying an annotation comment, the selected annotationdetermining unit 506 notifies the image processing unit 505 of adetecting result including annotation position information of theselected annotation.

Further, the image processing unit 505 changes the layer of a displayedimage in accordance with the order for jump display obtained from theselected annotation determining unit 506. Operations for changing thelayer of a displayed image in accordance with the order for jump displayas described above will be explained later as “operation of jumpdisplay”. The image processing unit 505 compounds a displayed image withan annotation comment in accordance with the order for displaying anannotation comment obtained from the selected annotation determiningunit 506. Operations for compounding an annotation comment in accordancewith the order for displaying an annotation comment as described abovewill be explained later as “operation of annotation comment display”.

[Operations of Information Processing Apparatus]

Next, a description will be given on operations of the PC 100. Thedescription will be given in the following order: (1) Operation ofannotation setting; (2) Operation of annotation mark display; (3)Operation of jump display; (4) Modified example of operation of jumpdisplay; and (5) Operation of annotation comment display.

[(1) Operation of Annotation Setting]

First, a description will be given on the operation of annotationsetting.

The image obtaining unit 501 obtains an image via a network or an imagestored in the removable recording medium 111, and stores the obtainedimage in the image storing unit 502.

The annotation setting unit 503 causes a user to select a display range(for example, one predetermined tile), the layer number of the Z-stack,that is, a position in the optical axis direction, and the like of animage to be displayed on the display screen, and obtains the selectingresult. Obtaining the selecting result, the annotation setting unit 503generates a control table 504 a in association with the display range ofthe image in the table storing unit 504. Next, as shown in FIG. 5, theannotation setting unit 503 registers the value of 1 to N eachrepresenting the position of the layer of the image including thedisplay range in the control table 504 a generated in the table storingunit 504 as the value of the Z number 504 d. Note that, in the controltable 504 a of FIG. 5, the values of 1 to 5 are stored as the values ofthe Z number 504 d.

Then, the annotation setting unit 503 notifies the image processing unit505 of information with regard to the display range and the value of theZ number included in the obtained selecting result. Based on theinformation with regard to the display range and the value of the Znumber in the notification, the image processing unit 505 obtains imagedata from the image storing unit 502 and generates display information,and expands the display information in the display buffer. The displayinformation stored in the display buffer is supplied to the display unit106. Based on the supplied display information, the display unit 106displays an image on the display screen.

An annotation can be set on an arbitrary portion in the displayed imagethrough the mouse or the keyboard of the input unit 107 by a user.Specifically, position information of the arbitrary portion is set asannotation position information by the user. The annotation setting unit503 obtains the annotation position information set through the input bythe user. Then, the annotation setting unit 503 registers the annotationposition information obtained through the input unit 107 as annotationposition information 504 b in relation with the value of the Z number504 d in the control table 504 a.

Then, the annotation setting unit 503 notifies the image processing unit505 of the registered annotation position information. Based on theannotation position information included in the notification, the imageprocessing unit 505 compounds an image G1 now displayed on the displayscreen with an annotation mark, which is obtained by visually reflectingthe annotation position information. Herein, the annotation mark may beany mark as long as a user can visually recognize a portion set with anannotation through display information, such as a predetermined-shapedframe. The image processing unit 505 expands the compound displayinformation (image data including annotation mark) in the displaybuffer. The display information stored in the display buffer is suppliedto the display unit 106. Accordingly, an image including an annotationmark is displayed on the display screen of the display unit 106.

Content information with regard to the set annotation can be registeredthrough the mouse or the keyboard of the input unit 107 by a user. Thecontent information with regard to an annotation is, for example, acomment with regard to the annotation by a user or the like. Theannotation setting unit 503 registers the content information withregard to an annotation input through the input unit 107 as annotationcontent information 504 c in relation with the value of the Z number 504d in the control table 504 a.

Further, an image of another layer can be displayed and an annotationcan be set with respect to an arbitrary portion in the newly-displayedimage through an operation of the mouse or the keyboard of the inputunit 107 by a user.

[(2) Operation of Annotation Mark Display]

Next, a description will be given on the operation of annotation markdisplay.

FIG. 6 is a flowchart showing the operation of annotation mark display.

The description will be given on the condition that the control table504 a is generated through (1) the operation of annotation setting.Further, in the set of the Z-stack of the display range in associationwith the control table 504 a, an image of a layer specified by apredetermined value of the Z number is displayed on the display screenof the display unit 106. Note that the image may include an annotationmark. That is, in a case where annotation position information isregistered in relation with the value of the Z number of the image inthe control table 504 a, an image including an annotation mark generatedbased on the annotation position information is displayed on the displayscreen. Meanwhile, in a case where annotation position information isnot registered in relation with the value of the Z number of the imagein the control table 504 a, an image including no annotation mark isdisplayed on the display screen.

In the state where an image including an annotation mark or an imageincluding no annotation mark is displayed on the display screen of thedisplay unit 106 as described above, the image processing unit 505monitors generation of an order to change a layer of a displayed imageinput through the input unit 107 by a user (hereinafter referred to as“focus change order”) (Step S101). Herein, the focus change order may beinput through operations of an input apparatus including a dial-typeknob connected to the PC 100. By using such an input apparatus includinga dial-type knob, a user can observe images while changing the layers ofthe displayed images as if the user observes an observation targetobject by changing focuses by rotating a focus dial of an actualmicroscope.

Obtaining the focus change order (Yes in Step S102), the imageprocessing unit 505 determines if the Z-through display mode is set ornot (Step S103). The Z-through display mode may be previously set by auser. Alternatively, the Z-through display mode may be set by a userwhen inputting the focus change order.

First, a description will be given on the case where the Z-throughdisplay mode is not set (No in Step S103).

The image processing unit 505 calculates the value “i” of the Z numberof a focus change target based on parameter information included in thefocus change order input through the input unit 107 such as a rotationamount, a rotation speed, and a rotation direction of the dial-typeknob, for example (Step S104). The image processing unit 505 obtainsimage data of the layer specified by the value “i” of the Z number fromthe image storing unit 502 (Step S105). Then, the image processing unit505 examines if annotation position information is registered or not inrelation with the value “i” of the Z number in the control table 504 a(Step S106). In a case where the annotation position information isregistered (Yes in Step S106), the image processing unit 505 compoundsthe image data obtained from the image storing unit 502 with anannotation mark, which is obtained by visually reflecting the annotationposition information, and expands the compound display information in adisplay buffer. Accordingly, an image including an annotation mark of alayer corresponding to the value “i” of the Z number is displayed on thedisplay screen (Step S107). After that, the image processing unit 505returns to the state of monitoring generation of a focus change orderinput through the input unit 107 by a user (Step S101).

Note that in a case where the annotation position information is notregistered in relation with the value “i” of the Z number in the controltable 504 a (No in Step S106), the image processing unit 505 merelyexpands image data in the display buffer. Accordingly, an imageincluding no annotation mark is displayed on the display screen of thedisplay unit 106 (Step S108). After that, the image processing unit 505returns to the state of monitoring generation of a focus change order(Step S101).

Next, a description will be given on the case where the Z-throughdisplay mode is set (Yes in Step S103).

Determining that the Z-through display mode is set, the image processingunit 505 calculates the value “i” of the Z number of the focus changetarget based on the parameter information included in the focus changeorder input through the input unit 107, and sets “0” as a default in a“c” being a variable indicating the number of layers having beensearched in the Z-stack (Step S109). Next, the image processing unit 505obtains image data of the layer specified by the value “i” of the Znumber of the focus change target from the image storing unit 502 (StepS110). Then, the image processing unit 505 examines if annotationposition information is registered or not in relation with the value “i”of the Z number in the control table 504 a (Step S111). In a case wherethe annotation position information is registered (Yes in Step S111),the image processing unit 505 compounds the image data obtained from theimage storing unit 502 with an annotation mark, which is obtained byvisually reflecting the annotation position information, and expands thecompound display information in a buffer. Accordingly, an imageincluding an annotation mark of a layer corresponding to the value “i”of the Z number is displayed on the display screen (Step S112). Then,the operation proceeds to Step S113. In a case where the annotationposition information is not registered (No in Step S111), the operationof compounding the annotation mark is skipped, and the operationproceeds to Step S113.

In Step S113, the image processing unit 505 increments the value “c”.After that, the image processing unit 505 judges whether the annotationposition information is searched in all the layers in the Z-stack inStep S105. That is, the image processing unit 505 judges if thecondition of c=z is established or not, “z” being the number of all thelayers in the Z-stack (Step S114). In a case where the condition of c=zis not established (No in Step S114), the image processing unit 505increments the value “i” of the Z number (Step S115). Next, the imageprocessing unit 505 examines if the value “i=i+1” of the Z numberobtained by the increment is larger than “z” or not (Step S116). Herein,a description will be given on a case where the value “i=i+1” is notlarger than “z”. In this case, the image processing unit 505 returns toStep S111, and examines if annotation position information is registeredor not in relation with the value “i+1” of the Z number obtained by theincrement. In a case where the annotation position information isregistered, the image processing unit 505 further compounds the compounddisplay information with an annotation mark, which is obtained byvisually reflecting the annotation position information (Step S112).Further, in a case where the value of the Z number of the focus changetarget is larger than “1”, before the annotation position information issearched in all the layers in the Z-stack, the value “i=i+1” becomeslarger than “z” in Step S116 (Yes in Step S116). In this case, the imageprocessing unit 505 sets “1” as the value “I” of the Z number (StepS117), and then, in a case where the annotation position information isregistered in relation with the value “1” of the Z number, furthercompounds the displayed image having been compounded with the annotationmark obtained by visually reflecting the annotation position information(Step S112). As a result, the operations of Step S111 and Step S112 arerepeated for z times. The annotation marks with regard to all theannotations set in the Z-stack are thus Z-through displayed. After that,the image processing unit 505 again returns to the state of monitoringgeneration of a focus change order input through the input unit 107 by auser (Step S101).

Herein, a description will be given on a specific example of this flow.The description will be given on the condition that the Z-throughdisplay mode is previously set by a user. Further, the control table 504a of FIG. 5 is generated through (1) the operation of annotationsetting. Further, the image G1 of the layer specified by the value “i=1”of the Z number registered in the control table 504 a is compounded withan annotation mark A1, which is obtained by visually reflecting theannotation position information registered in relation with the value“i=1” of the Z number, an annotation mark A3, which is obtained byvisually reflecting the annotation position information registered inrelation with the value “i=3” of the Z number, and an annotation markA5, which is obtained by visually reflecting the annotation positioninformation registered in relation with the value “i=5” of the Z number,to be displayed on the display screen (see, FIG. 7A). Then, in thisstate, “change focus to next layer” is input as a focus change order bya user.

In this case, based on the “change focus to next layer” order inputthrough the input unit 107, the image processing unit 505 calculates theZ value “i=2” of an image of the focus change target. The imageprocessing unit 505 obtains image data of the layer specified by thevalue “i=2” of the calculated Z number from the image storing unit 502.Subsequently, because annotation position information is not registeredin relation with the calculated value “i=2” of the Z number of the imageto be displayed next in the control table 504 a, the image processingunit 505 skips the operation of compounding the annotation mark. Afterthat, the image processing unit 505 increments the value “i=2” of the Znumber. Because annotation position information is registered inrelation with the calculated value “i=3” of the Z number in the controltable 504 a, the image processing unit 505 compounds the image data withan annotation mark A3, which is obtained by visually reflecting theannotation position information, and expands the compound displayinformation in a buffer. Accordingly, an image including the annotationmark A3 is displayed on the display screen (See FIG. 7B).

After that, in the similar manner, the image processing unit 505 changesthe value “i” of the Z number in the order of “4”, “5”, “1”. In eachcase, the image processing unit 505 further compounds the compounddisplay information with an annotation mark, which is obtained byvisually reflecting the annotation position information registered inrelation with the value “i” of the Z number after the change in thecontrol table 504 a. Then, the image processing unit 505 expands thecompound display information in the display buffer. Accordingly, animage including annotation marks is Z-through displayed on the displayscreen in a manner that the number of displayed annotation marks isincreased one by one (FIGS. 7C and 7D). After all the annotation marksare displayed as described above, the image processing unit 505 returnsto the state of monitoring generation of a focus change order inputthrough the input unit 107 by a user.

According to the Z-through display mode, even in a case where shapes andthe like of a tissue or a cell of an observation target object aredifferent in the optical axis direction of a microscope, it is possibleto compound one image with a plurality of annotations set on images of aplurality of different layers, respectively, to be displayed.Accordingly, in a case where an image on which an annotation is set andan image on which an annotation is not set are mixed in a Z-stack,annotation marks can be displayed on the image on which an annotation isnot set. So, irrespective of a speed of changing displayed images, it ispossible to avoid a risk of missing an annotation included in an imagewhich locates at the middle of the images by a user. Further, a usersuch as a pathologist can investigate a disorder carefully withreference to a plurality of annotation marks, and therefore a risk ofmissing a disorder is decreased.

Note that, in the above description, the image processing unit 505compounds an image with annotation marks, which are obtained by visuallyreflecting all the annotation position information registered in thecontrol table 504 a, but not limited to the above. The image processingunit 505 may compound an image with annotation marks, which are obtainedby visually reflecting some annotation position information out of allthe annotation position information registered in the control table 504a. Accordingly, in a case of displaying an image of a predeterminedlayer on the display screen, the image is compounded with an annotationmark, which is obtained by visually reflecting annotation positioninformation set on a different layer. Accordingly, even in a case wherea layer of a displayed image is changed, an image including anannotation mark set on a different layer can be displayed. As a result,it is possible to avoid a risk of missing an annotation mark by a user.

Further, in a case where the Z-through display mode is set, the imageprocessing unit 505 may compound an image with annotation marks in amanner that a positional relationship of layers of images on whichannotations are set is reflected. Accordingly, a user can intuitivelyunderstand the positional relationship in the optical axis directionbetween a layer of an image on which an annotation is set and a layer ofthe displayed image.

Specifically, in Step S105, the image processing unit 505 calculates thedifference between the value of the Z number specifying a layer of adisplayed image and the Z number registered in relation with annotationposition information of a displayed annotation mark in the control table504 a. As the value of the Z number, 1 to N each representing theposition of the layer of the image are registered in the control table504 a. So the difference of values of the Z number is smaller, thedistance between the layers is smaller. Further, the difference ofvalues of the Z number is larger, the distance between the layers islarger. By changing styles of displayed annotation marks based on thedifference of values of the Z number, the annotation marks can bedistinguishably displayed in a manner that a positional relationship inthe optical axis direction between a layer of an image on which anannotation is set and a layer of the displayed image are reflected. Forexample, the image processing unit 505 changes, in accordance with thedifference of values of the Z number, colors, brightness, chroma, alphablend values, widths of lines, styles of lines, and the like ofannotation marks. Accordingly, a user can intuitively understand thepositional relationship in the optical axis direction between a layer ofan image on which an annotation is set and a layer of the displayedimage. Specifically, the difference of values of the Z number is larger,the colors of annotation marks may be lighter. Alternatively, thedifference of values of the Z number is larger, the width of lines ofannotation marks may be thinner. Alternatively, the difference of valuesof the Z number is larger, the annotation marks may be more opaque.Alternatively, the difference of values of the Z number is larger, thealpha blend value may be more decreased to increase clarity ofannotation marks. According to the above-mentioned compound processing,a user can intuitively understand the positional relationship between alayer of the displayed image and layers of images including annotationmarks.

As described above, annotation marks are distinguishably displayed onthe display screen in a manner that a positional relationship in theoptical axis direction between images are reflected, a user canintuitively understand the positional relationship between a layer ofthe image displayed on the display screen and layers of images includingannotation marks. Further, the user can select an image to be displayedwith reference to the annotation marks. For example, in a case where animage including a plurality of annotation marks is displayed, a userwishes to cause an image of a layer on which one of the annotations isset to be displayed. The user can select the layer of the image to bedisplayed through the input unit 107 while paying attention to thechange of the styles of the annotation. So it is possible to change adisplayed image to an image of a predetermined layer easily andreliably. Accordingly, a risk of missing a disorder by a user such as apathologist is decreased.

[(3) Operation of Jump Display]

Next, a description will be given on the operation of jump display.

FIG. 9 is a flowchart showing the operation of jump display.

The description will be given on the condition that the Z-through modeis previously set by a user. Further, the control table 504 a isgenerated through (1) the operation of annotation setting. Further, inthe set of the Z-stack of the display range in association with thecontrol table 504 a, an image of a layer specified by a predeterminedvalue of the Z number is displayed on the display screen of the displayunit 106. Further, annotation marks set on a plurality of differentlayers are Z-through displayed through (2) the operation of annotationmark display.

In the state where an image including annotation marks is displayed onthe display screen as described above, the selected annotationdetermining unit 506 monitors generation of an order to change a layerof a displayed image input through the input unit 107 by a user(hereinafter referred to as “jump display order”) (Step S201). Herein,the jump display order may be input with an operation such as a fewclicks of the mouse of the input unit 107 by a user in a state where acursor is displayed in an area surrounded by an annotation mark includedin an image displayed on the display screen.

Obtaining the jump display order (Yes in Step S202), the selectedannotation determining unit 506 notifies the image processing unit 505of a detecting result including annotation position information of theselected annotation (Step S203).

Obtaining the detecting result including the annotation positioninformation from the selected annotation determining unit 506, the imageprocessing unit 505 cancels the Z-through display mode setting. Further,the image processing unit 505 obtains the value of the Z numberregistered in association with annotation position information includedin the detecting result in the control table 504 a. Obtaining the valueof the Z number, the image processing unit 505 obtains image data of alayer specified by the obtained value of the Z number as image data tobe displayed next from the image storing unit 502. The image processingunit 505 compounds the obtained image data with an annotation mark,which is obtained by visually reflecting the annotation positioninformation included in the detecting result obtained from the selectedannotation determining unit 506, and expands the compound displayinformation in the display buffer. Accordingly, an image including anannotation mark is displayed on the display screen (Step S204).

Herein, a description will be given on a specific example of this flow.The description will be given on the condition that the Z-throughdisplay mode is previously set by a user. Further, the control table 504a of FIG. 5 is generated through (1) the operation of annotationsetting. Further, an image G3 of the layer specified by the value “i=3”of the Z number registered in the control table 504 a is compounded withannotation marks A1, A3, A5, which are obtained by visually reflectingthe annotation position information registered in the control table 504a, to be Z-through displayed on the display screen. Then, in this state,the annotation mark A1 is selected by a user as the jump display order.

In this case, the image processing unit 505 obtains the jump displayorder (Yes in Step S202), and notifies the image processing unit 505 ofa detecting result including annotation position information of theselected annotation mark A1 (Step S203). Obtaining the detecting resultincluding the annotation position information from the selectedannotation determining unit 506, the image processing unit 505 cancelsthe Z-through display mode setting. Then, the image processing unit 505obtains the value “i=1” of the Z number registered in relation withannotation position information included in the detecting result in thecontrol table 504 a. Obtaining the value “i=1” of the Z number, theimage processing unit 505 obtains image data of a layer specified by theobtained value “i=1” of the Z number as image data to be displayed nextfrom the image storing unit 502. The image processing unit 505 compoundsthe obtained image data with the annotation mark A1, which is obtainedby visually reflecting annotation position information included in thedetecting result obtained from the selected annotation determining unit506, and expands the compound display information in the display buffer.Accordingly, as shown in FIG. 10, the image G1 including the annotationmark A1 is displayed on the display screen (Step S204).

According to the jump display function, through input operation withrespect to an annotation mark, the display can be switched over to animage on which the annotation is set. For example, in a case where adisorder may exist in an image of the displayed layer, by changing overthe display to an image of a different layer, a user can confirm ifthere is a disorder or not again. Accordingly, a user such as apathologist can investigate a disorder more carefully, and therefore arisk of missing a disorder is decreased.

In the above description, in Step 5204, the image processing unit 505cancels the Z-through display mode setting, but not limited to theabove. Obtaining the detecting result including the annotation positioninformation from the selected annotation determining unit 506, the imageprocessing unit 505 may compound the image data obtained from the imagestoring unit 502 with a plurality of annotation marks having beendisplayed and may expand the compound result in the display bufferwithout canceling the Z-through display mode setting. Note that the sameis equally true of a case where the phrase such as “to cancel theZ-through display mode setting” is used in the following description.

Accordingly, display can be changed over repeatedly between images of aplurality of different layers on which annotations are set,respectively. So images of a plurality of different layers can bedisplayed alternately. For example, in a case where there may be adisorder in an image of the displayed layer, an image of a differentlayer is displayed and a user can confirm if there is the disorder ornot again, and after that, the image which was displayed at first isdisplayed again and the user can confirm if there is the disorder or notagain. Accordingly, a user such as a pathologist can investigate adisorder more carefully, and therefore a risk of missing a disorder isdecreased.

[(4) Modified Example of Operation of Jump Display]

Next, a description will be given on a modified example of the operationof jump display. In this modified example, a display processing isperformed such that the selected annotation mark is arranged on anapproximately center portion of an image displayed on the displayscreen.

FIG. 11 is a flowchart showing the modified example of the operation ofjump display

The description will be given on the condition that the Z-through modeis previously set by a user. Further, the control table 504 a isgenerated through (1) the operation of annotation setting. Further, inthe set of the Z-stack of the display range in association with thecontrol table 504 a, an image of a layer specified by a predeterminedvalue of the Z number is displayed on the display screen of the displayunit 106. Further, a plurality of annotation marks are Z-throughdisplayed through (2) the operation of annotation mark display.

In the state where an image including annotation marks is displayed onthe display screen, the selected annotation determining unit 506performs the processing of Step S201 to Step S203 (Step S301). That is,in the state where an image including annotation marks is displayed onthe display screen, the selected annotation determining unit 506monitors generation of a jump display order input through the input unit107 by a user (Step S201).

Obtaining the jump display order (Yes in Step S202), the selectedannotation determining unit 506 notifies the image processing unit 505of a detecting result including annotation position information of theselected annotation (Step S203).

Obtaining the detecting result including the annotation positioninformation from the selected annotation determining unit 506, the imageprocessing unit 505 obtains the value of the Z number registered inassociation with annotation position information included in thedetecting result in the control table 504 a.

Obtaining the annotation position information from the selectedannotation determining unit 506, the image processing unit 505 cancelsthe Z-through display mode setting. Then, the image processing unit 505sets the annotation position information included in the detectingresult as a reference position, and determines a display range of animage based on the reference position. Specifically, the imageprocessing unit 505 determines a display range of an image in which anannotation mark specified by the annotation position information isdisplayed on an approximately center portion of the image, in theoriginally-displayed image (Step S302). The image processing unit 505notifies the annotation setting unit 503 of the determined displayrange. The annotation setting unit 503 generates a control tablecorresponding to the display range obtained from the image processingunit 505, and stores the control table in the table storing unit 504(Step S303).

Then, the image processing unit 505 obtains image data of a layerspecified by the value of the Z number obtained from the control tableand the position information with regard to the display range in theoriginally-displayed image calculated in Step S302, as image data to bedisplayed next from the image storing unit 502. The image processingunit 505 compounds the obtained image data with an annotation mark,which is obtained by visually reflecting the annotation positioninformation in the detecting result obtained from the selectedannotation determining unit 506, and expands the compound displayinformation in the display buffer. Accordingly, an image including anannotation mark arranged on the approximately center portion of theimage is displayed on the display screen (Step S304).

Accordingly, since an annotation mark can be displayed on anapproximately center portion of a displayed image, a user can easilyobserve a portion in the vicinity of the portion on which the annotationmark is displayed. For example, in a case where there is a fear that adisorder may exist in the vicinity of a disorder in the portion on whichthe annotation mark is displayed, a user such as a pathologist canexamine the disorder more carefully and a risk of missing a disorder isdecreased.

[(5) Operation of Annotation Comment Display]

Next, a description will be given on the operation of annotation commentdisplay.

FIG. 12 is a flowchart showing the operation of annotation commentdisplay.

The description will be given on the condition that the control table504 a is generated through (1) the operation of annotation setting.Further, in the set of the Z-stack of the display range in associationwith the control table 504 a, an image of a layer specified by apredetermined value of the Z number is displayed on the display screenof the display unit 106. Further, an annotation mark or annotation marksis/are displayed through (2) the operation of annotation mark display.

In the state where an image including an annotation mark or annotationmarks is displayed on the display screen as described above, theselected annotation determining unit 506 monitors generation of an orderto display an annotation comment input through the input unit 107 by auser (hereinafter referred to as “annotation comment display order”)(Step S401). Herein, the annotation comment display order may be inputwith an operation such as a few clicks of the mouse of the input unit107 by a user in a state where a cursor is displayed in an areasurrounded by an annotation mark included in an image displayed on thedisplay screen.

Obtaining the annotation comment display order (Yes in Step S402), theselected annotation determining unit 506 notifies the image processingunit 505 of a detecting result including annotation position informationof the selected annotation (Step S403).

Obtaining the detecting result including the annotation positioninformation from the selected annotation determining unit 506, the imageprocessing unit 505 cancels the Z-through display mode setting in a casewhere the Z-through display mode setting has been set. Then, the imageprocessing unit 505 obtains annotation content information registered inrelation with the annotation position information included in thedetecting result in the control table 504 a. Obtaining the annotationcontent information, the image processing unit 505 compounds the imageincluding an annotation mark displayed on the display screen with anannotation comment, which is obtained by visually reflecting theobtained annotation content information, and expands the compounddisplay information in the display buffer. Accordingly, an imageobtained by compounding an image including an annotation mark with anannotation comment is displayed on the display screen (Step S404).

Herein, a description will be given on a specific example of this flow.The description will be given on the condition that the control table504 a is generated through (1) the operation of annotation setting.Further, in the set of the Z-stack of the display range in associationwith the control table 504 a, the image G1 of the layer specified by thevalue “i=1” of the Z number is displayed on the display screen of thedisplay unit 106. Further, an annotation mark A1 or a plurality ofannotation marks including the annotation mark A1 is/are displayed onthe display screen through (2) the operation of annotation mark display.Then, in this state, the annotation mark A1 is selected by a user as theannotation comment display order.

In this case, the selected annotation determining unit 506 obtains theannotation comment display order (Yes in Step S402), and notifies theimage processing unit 505 of a detecting result including annotationposition information of the selected annotation mark A1 (Step S403).Obtaining the detecting result including the annotation positioninformation from the selected annotation determining unit 506, the imageprocessing unit 505 cancels the Z-through display mode setting in a casewhere the Z-through display mode setting has been set. Then, the imageprocessing unit 505 obtains annotation content information “Disorder”registered in relation with the annotation position information includedin the detecting result in the control table 504 a. Obtaining theannotation content information “Disorder”, the image processing unit 505compounds the image G1 including the annotation mark A1 displayed on thedisplay screen with an annotation comment, which is obtained by visuallyreflecting the obtained annotation content information “Disorder”, andexpands the compound display information in the display buffer.Accordingly, as shown in FIG. 8, an image obtained by compounding theimage G1 including the annotation mark A1 with the annotation comment isdisplayed on the display screen (Step S404).

According to the annotation comment display function, an annotation canbe readably displayed. So the annotation comment display function isuseful in a case of performing an examination, a diagnosis, or the likewhile displaying content information with regard to a portion to which auser paid attention before, or in a case where a user wishes to reviewcontent information which was set by another user, for example.

The operation of annotation mark display, the operation of jump display,and the operation of annotation comment display can be on/off by usingthe input unit 107.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

In the above description, as the values each representing the positionof the layer of the image, the values “i=1 to N” of the Z number areregistered in the control table 504 a, but not limited to the above. Asa value representing the position of the layer of the image, theposition of the layer in the optical axis direction, that is, a focallength may be registered. For example, instead of the values “i=1” to“i=5” of the Z numbers as shown in FIG. 5, “500 μm”, “502 μm”, “504 μm”,“506 μm”, and “508 μm” may be registered as the positions of the layersof images in the optical axis direction. In this case, in (2) theoperation of annotation mark display, the image processing unit 505calculates the focal length of an image to be displayed next based onparameter information included in the focus change order input throughthe input unit 107. The image processing unit 505 obtains image data ofa layer specified by the calculated focal length from the image storingunit 502. Then, the image processing unit 505 examines if annotationposition information is registered or not in relation with thecalculated focal length in the control table 504 a. In a case whereannotation position information is registered, the image processing unit505 compounds the image data obtained from the image storing unit 502with an annotation mark, which is obtained by visually reflecting theannotation position information, and expands the compound displayinformation in the display buffer. Accordingly, an image including anannotation mark of a layer corresponding to the calculated focal lengthis displayed on the display screen (Step S104).

In the above, the mode is described in which the image data that formsthe image pyramid structure 50 is stored in the storage unit 108 of thePC 100. However, instead of the PC 100, another computer or a server maystore the image data that forms the image pyramid structure 50, and thePC 100 used by the user as a terminal apparatus may access the computeror the server to receive the image data. In this case, the PC 100 as theterminal apparatus and the server or the like may be connected via anetwork such as a LAN and a WAN. In particular, the use of the WAN canrealize telepathology, telediagnosis, or the like.

It should be understood that various changes and modifications to thepresently preferred embodiments described herein will be apparent tothose skilled in the art. Such changes and modifications can be madewithout departing from the spirit and scope and without diminishing itsintended advantages. It is therefore intended that such changes andmodifications be covered by the appended claims.

The application is claimed as follows:
 1. An information processing apparatus comprising: a processor; and a memory device operatively coupled to the processor, the memory device storing instructions that cause the processor, in cooperation with the memory device, to cause a display device to display: (a) a first image from images associated with an observation target object, the images including the first image and a second image; and (b) a first annotation mark corresponding to the second image, the first annotation mark overlapping the first image.
 2. The information processing apparatus of claim 1, wherein the images are obtained by a microscope.
 3. The information processing apparatus of claim 1, wherein the observation target object includes a section of biological tissue.
 4. The information processing apparatus of claim 1, wherein the first image and the second image have a first resolution.
 5. The information processing apparatus of claim 4, wherein the instructions, when executed by the processor, cause the processor to enable a user to change the first resolution of the first image and the second image to a second, different resolution.
 6. The information processing apparatus of claim 1, wherein the images correspond to a plurality of different positions in an optical axis direction.
 7. The information processing apparatus of claim 6, wherein the plurality of different positions in the optical axis direction are determined based on focusing positions.
 8. The information processing apparatus of claim 1, wherein the first annotation mark includes a predetermined shaped frame.
 9. The information processing apparatus of claim 1, wherein the instructions, when executed by the processor, cause the processor to display an annotation comment overlaying the displayed first image, the annotation comment being different from the displayed first annotation mark.
 10. The information processing apparatus of claim 1, wherein: (a) an input device is operatively coupled to the processor; and (b) the instructions, when executed by the processor, cause the processor to operate with the input device to enable a user to input annotation information.
 11. The information processing apparatus of claim 1, wherein: (a) the first annotation mark corresponds to a position on the second image; and (b) the instructions, when executed by the processor, cause the processor to display the first annotation mark at a position on the first image which corresponds to the position of the second image.
 12. The information processing apparatus of claim 1, wherein the instructions, when executed by the processor, cause the processor to display a second annotation mark corresponding to the first image.
 13. The information processing apparatus of claim 12, wherein: (a) the first image corresponds to a first layer which corresponds to a first position in an optical axis direction; and (b) the second image corresponds to a second layer which corresponds to a second position in the optical axis direction.
 14. The information processing apparatus of claim 13, wherein: (a) the first annotation mark and the second annotation mark have at least one different color, brightness, chroma, alpha blend values, widths of lines, and styles of lines; and (b) the instructions, when executed by the processor, cause the processor to change at least one of the color, the chroma, the alpha blend values, the widths of lines, and the styles of lines based on said layers which correspond to said positions in the optical axis direction.
 15. The information processing apparatus of claim 12, wherein the instructions, when executed by the processor, cause the processor to: (a) enable a user to select the first annotation mark; and (b) in response to the first annotation mark being selected, display the second image, wherein the first annotation mark overlaps the displayed second image.
 16. The information processing apparatus of claim 12, wherein the instructions, when executed by the processor, cause the processor to: (a) enable a user to select one of the first annotation mark and the second annotation mark; (b) in response to the first annotation mark being selected, cancel the display of the second annotation mark; and (c) in response to the second annotation mark being selected, cancel the display of the first annotation mark.
 17. The information processing apparatus of claim 15, wherein the instructions, when executed by the processor, cause the processor to, in response to the first annotation mark being selected, arrange and display the first annotation mark on a center portion of the displayed second image.
 18. The information processing apparatus of claim 1, wherein the instructions, when executed by the processor, cause the processor to: (a) determine whether a user has requested a change from displaying the first image to displaying the second image; (b) in response to the change being requested, determine whether a mode is set; and (c) in response to the mode being set, display: the second image; and (ii) the first annotation mark overlapping the second image.
 19. The information processing apparatus of claim 18, wherein the instructions, when executed by the processor, cause the processor to, in response to the mode being set, display a third annotation mark overlapping the second image, the third annotation mark corresponding to a third image.
 20. The information processing apparatus of claim 18, wherein the instructions, when executed by the processor, cause the processor to, in response to the mode not being set, display: (a) the second image; and (b) the first annotation mark overlapping the second image. 